This is how language models usually start off: They guess randomly and produce gibberish. Over many, many rounds of training, language models can learn to write. As language models grow larger, the patterns that they learn can become increasingly complex. More is differentOther than the additional fine-tuning stages, the primary difference between nanoGPT and the language model underlying chatGPT is size. Large language models can become highly unpredictable, as evidenced by Microsoft Bing A.I.’s early interactions with my colleague Kevin Roose.

---